Methods and systems for automatic fullness estimation of containers

ABSTRACT

A method and apparatus for (a) receiving a three-dimensional (3D) point cloud from a depth sensor that is oriented towards an open end of a shipping container, the point cloud comprising a plurality of points that each have a respective depth value, (b) segmenting the received 3D point cloud among a plurality of grid elements, (c) calculating a respective loaded-container-portion grid-element volume for each grid element, (d) calculating a loaded-container-portion volume of the shipping container by aggregating the calculated respective loaded-container-portion grid-element volumes, (e) calculating an estimated fullness of the shipping container based on the loaded-container-portion volume and a capacity of the shipping container; and (f) outputting the calculated estimated fullness of the shipping container.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/944,860, filed Nov. 18, 2015, entitled “Methods and Systems ForAutomatic Fullness Estimation of Containers,” which is incorporated byreference herein in its entirety.

BACKGROUND OF THE INVENTION

Efficient loading of containers is a key element to successfuldistribution in the transportation and logistics industry. Ensuring thateach container is loaded efficiently throughout the loading process isvital to successful distribution. However, the inability to verify thateach container meets this goal has been a problem in the industry.

There is a need for real-time monitoring or measurements of thecontainers during the loading process. This functionality could providegood business value to vendors through loading optimization.

Accordingly, there is a need for methods and systems for automaticfullness estimation of containers.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying figures, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed invention, and explainvarious principles and advantages of those embodiments.

FIG. 1 depicts a shipping container, in accordance with someembodiments.

FIG. 2A depicts a flat back surface of a shipping container, accordancewith some embodiments.

FIG. 2B depicts a curved back surface of a shipping container, inaccordance with some embodiments.

FIG. 3 depicts a loaded-container point cloud, in accordance with someembodiments.

FIG. 4 depicts a segmented loaded-container point cloud, in accordancewith some embodiments.

FIG. 5 depicts an expanded-grid-element view of a segmentedloaded-container point cloud, in accordance with some embodiments.

FIG. 6 depicts an architectural view of an example computing device, inaccordance with some embodiments.

FIG. 7 depicts an example method, in accordance with some embodiments.

FIG. 8 depicts a shipping container having an optically readableidentifier in accordance with some embodiments.

Skilled artisans will appreciate that elements in the figures areillustrated for simplicity and clarity and have not necessarily beendrawn to scale. For example, the dimensions of some of the elements inthe figures may be exaggerated relative to other elements to help toimprove understanding of embodiments of the present invention.

The apparatus and method components have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present invention so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

DETAILED DESCRIPTION OF THE INVENTION

One embodiment takes the form of a process that includes (a) receiving athree-dimensional (3D) point cloud from a depth sensor that is orientedtowards an open end of a shipping container, where the point cloudincludes a plurality of points that each have a respective depth value,(b) segmenting the received 3D point cloud among a plurality of gridelements, (c) calculating a respective loaded-container-portiongrid-element volume for each grid element, (d) calculating aloaded-container-portion volume of the shipping container by aggregatingthe calculated respective loaded-container-portion grid-element volumes,(e) calculating an estimated fullness of the shipping container based onthe loaded-container-portion volume and a capacity of the shippingcontainer; and (f) outputting the calculated estimated fullness of theshipping container.

A second embodiment takes the form of a system that includes a depthsensor, a communication interface, a processor, a data storagecontaining instructions executable by the processor for causing hesystem to carry out at least the functions described in the precedingparagraph.

In at least one embodiment, the plurality of grid elements collectivelyforms a two-dimensional (2D) grid image that corresponds to a plane thatis parallel to the open end of the shipping container, where each gridelement has a respective grid-element area, and the method furtherincludes determining a respective loaded-container-portion grid-elementdepth value for each grid element, where calculating the respectiveloaded-container-portion grid-element volume for each grid element isbased on at least the respective grid-element area and the respectiveloaded-container-portion grid-element depth value for each respectivegrid element.

In at least one embodiment, the method further includes determining anunloaded-container-portion depth value for each grid element, anddetermining a respective loaded-container-portion grid-element depthvalue for each grid element is based at least in part on the differencebetween (i) a depth dimension of the shipping container and (ii) thedetermined unloaded-container-portion depth value for the correspondinggrid element.

In at least one embodiment, assigning the grid-element depth value forthe given grid element based on the depth values of the points in thepoint cloud that correspond to the given grid element includes assigningas the grid-element depth value for the given grid element a minimumvalue from among the depth values of the points in the point cloud thatcorrespond to the given grid element.

In at least one embodiment, assigning the grid-element depth value forthe given grid element based on the depth values of the points in thepoint cloud that correspond to the given grid element includes assigningas the grid-element depth value for the given grid element an averagevalue of the depth values of the points in the point cloud thatcorrespond to the given grid element.

In at least one embodiment, the depth dimension of the shippingcontainer is a grid-element-specific depth dimension that is based on acorresponding grid element in a reference empty-container point cloud.In at least one such embodiment, the reference empty-container pointcloud reflects a back wall of the shipping container being a flatsurface. In at least one other such embodiment, the referenceempty-container point cloud reflects a back wall of the shippingcontainer being a curved surface.

In at least one embodiment, the method further includes cleaning up the2D grid image prior to determining a respective loaded-container-portiongrid-element depth value for each grid element.

In at least one embodiment, the depth sensor has an optical axis and animage plane, and the method further includes, prior to segmenting thereceived point cloud among the plurality of grid elements, rotating thereceived 3D point cloud to align (i) the optical axis with a groundlevel and (ii) the image plane with an end plane of the shippingcontainer.

In at least one embodiment, rotating the point cloud is based on anoffline calibration process using the ground level and the end plane asreference.

In at least one embodiment, the method further includes determining thecapacity of the shipping container based at least in part on thereceived 3D point cloud.

In at least one embodiment, the method further includes (i) receiving anoptical image of the shipping container and (ii) determining thecapacity of the shipping container based at least in part on thereceived optical image. In at least one such embodiment, determining thecapacity of the shipping container based at least in part on thereceived optical image includes (i) determining at least one physicaldimension of the shipping container from the received optical image and(ii) determining the capacity of the shipping container based on the atleast one determined physical dimension. In at least one other suchembodiment, determining the capacity of the shipping container based atleast in part on the received optical image includes (i) using opticalcharacter recognition (OCR) on the at least one received optical imageto ascertain at least one identifier of the shipping container and (ii)using the at least one ascertained identifier of the shipping containerto determine the capacity of the shipping container.

In at least one embodiment, each grid element has sides substantiallyequal to 5 millimeters (mm) in length.

In at least one embodiment, each grid element is substantially square inshape, and a grid-element side length is an adjustable parameter.

Moreover, any of the variations and permutations described herein can beimplemented with respect to any embodiments, including with respect toany method embodiments and with respect to any system embodiments.Furthermore, this flexibility and cross-applicability of embodiments ispresent in spite of the use of slightly different language (e.g.,process, method, steps, functions, set of functions, and the like) todescribe and or characterize such embodiments.

Before proceeding with this detailed description, it is noted that theentities, connections, arrangements, and the like that are depictedin—and described in connection with—the various figures are presented byway of example and not by way of limitation. As such, any and allstatements or other indications as to what a particular figure“depicts,” what a particular element or entity in a particular figure“is” or “has,” and any and all similar statements—that may in isolationand out of context be read as absolute and therefore limiting—can onlyproperly be read as being constructively preceded by a clause such as“In at least one embodiment, . . . .” And it is for reasons akin tobrevity and clarity of presentation that this implied leading clause isnot repeated ad nauseum in this detailed description.

FIG. 1 depicts a shipping container, in accordance with someembodiments. In particular, FIG. 1 depicts (i) a shipping container 102and (ii) a depth sensor 104 that is oriented towards an open end of theshipping container 102. In various different examples, the shippingcontainer 102 could be designed for travel by truck, rail, boat, plane,and/or any other suitable mode or modes of travel. Moreover, as is morefully discussed herein, the shipping container 102 could have any of anumber of different shapes; a substantially rectangular shape (i.e., arectangular cylinder) is depicted by way of example in FIG. 1. Asdepicted in FIG. 1, the shipping container 102 contains objects (e.g.,boxes and/or other packages) 106. The shipping container 102 may have anumber of different surfaces, perhaps flat, perhaps curved, amongnumerous other possibilities that could be listed here.

There are a number of types of depth sensor 104 that could be used,perhaps one that includes an RGB sensor, perhaps leap motion, perhapsIntel perceptual computing, perhaps Microsoft Kinect, among numerousother possibilities that could be listed here. There are also a numberof depth-sensing techniques that could be implemented by the depthsensor 104, perhaps using stereo triangulation, perhaps using time offlight, perhaps using coded aperture, among numerous other possibilitiesthat could be listed here. As one example, the depth sensor 104 could bemounted to a wall or column or the like in a given shipping warehouse,and the shipping container 102 could be positioned on the back of atruck, and then driven (e.g., backed) into a position such that thedepth sensor 104 is oriented towards an open end of the shippingcontainer 102, as is depicted in FIG. 1.

As mentioned above, different shipping containers could have differentshapes. Two examples are shown in FIGS. 2A and 2B. In particular, FIG.2A depicts (i) a flat back wall (i.e., surface) 202 of a shippingcontainer and (ii) a depth sensor 204, whereas FIG. 2B depicts (i) acurved back wall (i.e., surface) 206 of a shipping container and (ii) adepth sensor 208. And certainly numerous other examples ofshipping-container shapes could be presented here.

FIG. 3 depicts a loaded-container point cloud, in accordance with someembodiments. In particular, FIG. 3 depicts a 3D point cloud 302. As ageneral matter, the depth sensor that is oriented at the open end of theshipping container may gather depth information in a given field of viewand transmit that information to a system that may be equipped,programmed, and configured to carry out the present systems and methods.That set of information (i.e., points) is referred to herein as being a3D point cloud (or at times simply a point cloud); each point in such acloud corresponds to a perceived depth at a corresponding point in thefield of view of the depth sensor.

Returning to FIG. 3, an outline 304 of a shipping container is shown, asare outlines 306A, 306B, and 306C of example packages in the exampleshipping container. These outlines 304 and 306A-C are intended togenerally correspond to the shipping container 104 and the packages 106that are depicted in FIG. 1, in order to help the reader visualize anexample real-world scenario from which the example point cloud 302 couldhave been derived, gathered, or the like. Moreover, for purposes ofillustration, each point in the point cloud 302 is shown in FIG. 3 ashaving an integer number that corresponds to an example depth value (in,e.g., example units such as meters). In actual implementations, anynumber of points could be present in the point cloud 302, as the variouspoints that are depicted in FIG. 3 as being part of the point cloud 302are for illustration and are not meant to be comprehensive.

Moreover, as is more fully discussed below, in some embodiments thedepth sensor that is oriented towards an open end of the shippingcontainer has a vantage point with respect to the open end of theshipping container that is not aligned with the center of the open endof the shipping container in one or more dimensions. That is, the depthsensor and the shipping container might be relatively positioned suchthat the depth sensor is looking to some extent from one side or theother and could be vertically off center (e.g., elevated) as well. So,for example, the depth sensor may be positioned higher and to the rightof the center of the plane that corresponds with the open end of theshipping container.

As is more fully described below, the present disclosure includessegmentation of the received point cloud into a number of grid elementsthat collectively correspond to the open end of the shipping container.In cases where the depth sensor happens to be positioned square to theopen end of the shipping container and vertically centered on that openend as well, this segmentation step can be proceeded to without firstperforming one or more geometric rotations and/or projections. In othercases, however, prior to carrying out the below-described segmentationstep and the various other steps that are subsequent to that, thepresent systems and methods include a step of projecting the receivedpoint cloud on to a two-dimensional (2D) grid that corresponds to theopen end of the shipping container using one or more geometric rotationsand/or projections in accordance with the relative positions of thedepth sensor and the open end of the shipping container. Such relativeposition can be pre-programmed into the system, or could otherwise bedetermined using depth sensors, optical cameras, and/or other suitableequipment.

FIG. 4 depicts a segmented loaded-container point cloud, in accordancewith some embodiments. In particular, FIG. 4 depicts a segmented 3Dpoint cloud 402, which may be generated in a number of different ways,such as edge-based segmentation, surfaced-based segmentation, and/orscanline-based segmentation, among numerous other possibilities that maybe listed here. Moreover, it is noted that FIG. 4 depicts the segmentedpoint cloud 402 after any necessary rotations and/or projections wereperformed to account for the relative positions and alignments of thedepth sensor and the open end of the shipping container.

As described above, in at least one embodiment, the point cloud 402 issegmented among a plurality of grid elements, which collectively form a2D grid image that corresponds to a plane that is parallel to the openend of the shipping container. Each grid element has a respectivegrid-element area. In FIG. 4, the grid elements are shown as beingsubstantially square (e.g., 5 mm by 5 mm), though this is by way ofexample and not limitation, as any suitable dimensions and/or shapescould be used as deemed suitable by those of skill in the art for agiven implementation. Moreover, in some embodiments, the side length ofthe grid elements is an adjustable parameter. In some cases, thisparameter is set to be as small a value as the associated depth sensorallows and/or is capable of. Indeed, the resolution of the depth sensorplays a role in whether estimates of container fullness areoverestimates or underestimates. As can be seen in FIG. 4, one examplegrid element 404 is highlighted by way of example. The grid element 404is depicted as including ten total points from the segmented point cloud402; four of those ten points have a depth value of 1 (e.g., 1 meter),five of those ten points have a depth value of 2 (e.g., 2 meters), andone of those ten points has a depth value of 3 (e.g., 3 meters). Thisnumber of points in grid element 404 and these respective depth valuesare provided purely by way of example and for illustration, and in noway for limitation.

FIG. 5 depicts an expanded-grid-element view of a segmentedloaded-container point cloud, in accordance with some embodiments. Inparticular, FIG. 5 depicts a segmented 3D point cloud 502 (though zoomedout too far to depict individual points) and an expanded grid element504. The expanded grid element 504 includes, by way of example only, thesame set of ten points that are in the grid element 404 of FIG. 4,albeit in a different arrangement; i.e., there are ten total points,including four points having a depth value of 1, five points having adepth value of 2, and one point having a depth value of 3.

In connection with various embodiments, the grid element 504 is assigneda characteristic depth value based on the depth values of the points inthe subsection of the 3D point cloud that is found in the particulargrid element 504. From among those depth values, the characteristicdepth value for the grid element could be a minimum value, a mode (i.e.,most commonly occurring) value, an average value, or some otherpossibility. Using the example data that is present in FIG. 5: if theminimum value were used, then the characteristic depth value for thegrid element 504 would be 1; if the mode value were used, then thecharacteristic depth value for the grid element 504 would be 2; if theaverage value were used, then the characteristic depth value for thegrid element 504 would be 1.7 (or 2 if rounded to the nearest wholenumber). And certainly numerous other possible implementations could belisted here. As is described more fully below, the characteristic depthvalue that is assigned to a given grid element is then used, along withthe area of that grid element, to calculate a loaded-portion volume forthat particular grid element.

FIG. 6 depicts an architectural view of an example computing device, inaccordance with some embodiments. The example computing device 600 maybe configured to carry out the functions described herein, and asdepicted includes a communications interface 602, a processor 604, datastorage 606 (that contains program instructions 608 and operational data610), a user interface 612, peripherals 614, and a communication bus616. This arrangement is presented by way of example and not limitation,as other example arrangements could be described here.

The communication interface 602 may be configured to be operable forcommunication according to one or more wireless-communication protocols,some examples of which include LMR, LTE, APCO P25, ETSI DMR, TETRA,Wi-Fi, Bluetooth, and the like. The communication interface 602 may alsoor instead include one or more wired-communication interfaces (forcommunication according to, e.g., Ethernet, USB, and/or one or moreother protocols.) The communication interface 602 may include anynecessary hardware (e.g., chipsets, antennas, Ethernet interfaces,etc.), any necessary firmware, and any necessary software for conductingone or more forms of communication with one or more other entities asdescribed herein.

The processor 604 may include one or more processors of any type deemedsuitable by those of skill in the relevant art, some examples includinga general-purpose microprocessor and a dedicated digital signalprocessor (DSP).

The data storage 606 may take the form of any non-transitorycomputer-readable medium or combination of such media, some examplesincluding flash memory, read-only memory (ROM), and random-access memory(RAM) to name but a few, as any one or more types of non-transitorydata-storage technology deemed suitable by those of skill in therelevant art could be used. As depicted in FIG. 6, the data storage 606contains program instructions 608 executable by the processor 604 forcarrying out various functions described herein, and further is depictedas containing operational data 610, which may include any one or moredata values stored by and/or accessed by the computing device incarrying out one or more of the functions described herein.

The user interface 612 may include one or more input devices (a.k.a.components and the like) and/or one or more output devices (a.k.a.components and the like.) With respect to input devices, the userinterface 612 may include one or more touchscreens, buttons, switches,microphones, and the like. With respect to output devices, the userinterface 612 may include one or more displays, speakers, light emittingdiodes (LEDs), and the like. Moreover, one or more components (e.g., aninteractive touchscreen and display) of the user interface 612 couldprovide both user-input and user-output functionality.

The peripherals 614 may include any computing device accessory,component, or the like, that is accessible to and useable by thecomputing device 600 during operation. In some embodiments, theperipherals 614 includes a depth sensor. In some embodiments, theperipherals 614 includes a camera for capturing digital video and/orstill images. And certainly other example peripherals could be listed.

FIG. 7 depicts an example method, in accordance with some embodiments.In particular, FIG. 7 depicts a method 700 that includes steps 702, 704,706, 708, 710, and 712, and is described below by way of example asbeing carried out by the computing system 600 of FIG. 6, though ingeneral the method 700 could be carried out by any computing device thatis suitably equipped, programmed, and configured.

At step 702, the computing system 600 receives a 3D point cloud from adepth sensor that is oriented towards an open end of a shippingcontainer. The point cloud includes a plurality of points that each havea respective depth value. As described above, if necessary due to therespective positioning and alignment of the depth sensor and the openend of the shipping container, the computing system 600, upon receivingthe 3D point cloud, may rotate the received 3D point cloud to align (i)an optical axis of the depth sensor with a ground level and (ii) animage plane of the depth sensor with an end plane of the shippingcontainer. This rotating of the received point cloud may be based on acalibration process (e.g., an offline calibration process) that uses theground level and the end plane as reference.

At step 704, the computing system 600 segments the 3D point cloud thatwas received at step 702 among a plurality of grid elements. Asdescribed above, those grid elements could be substantially rectangular(e.g., square) in shape, and they may collectively form a 2D grid imagethat corresponds to a plane that is parallel to the open end of theshipping container, where each grid element has a respectivegrid-element area.

At step 706, the computing system 600 calculates a respectiveloaded-container-portion grid-element volume for each grid element. Thecomputing system 600 may do so by first determining a respectiveloaded-container-portion grid-element depth value for each grid element,and then determining each respective loaded-container-portiongrid-element volume for each grid element by multiplying the particulargrid element's area by the particular grid element's respectiveloaded-container-portion grid-element depth value. In some embodiments,the computing system 600 cleans up the 2D grid image prior todetermining a respective loaded-container-portion grid-element depthvalue for each grid element.

As to how the computing system 600 may determine a particular gridelement's respective loaded-container-portion grid-element depth value,in one embodiment the computing system 600 determines anunloaded-container-portion depth value for the particular grid element,and then determines the respective loaded-container-portion grid-elementdepth value for the particular grid element is based at least in part onthe difference between (i) a depth dimension of the shipping containerand (ii) the determined unloaded-container-portion depth value for thecorresponding grid element. Thus, for example, if the computing system600 determined that the unloaded-container-portion depth value of agiven grid element was 3 meters and knew that the depth dimension of theshipping container was 50 meters, the computing system 600 coulddetermine that the loaded-container-portion depth value for the givengrid element was 47 meters.

As to how the computing system 600 may determine theunloaded-container-portion depth value for a given grid element, in someembodiments the computing system 600 assigns a characteristicgrid-element depth value to the given grid element based on the depthvalues of the points in the point cloud that correspond to the givengrid element. As described above, some options for doing so includingselecting a minimum value, a mode value, and an average value. A maximumvalue could also be selected, those this would tend to lead tounderloading of containers by overestimating their fullness, which wouldbe less than optimally efficient.

Upon assigning a characteristic grid-element depth value to the givengrid element, the computing system 600 may then determine the respectiveunloaded-container-portion depth value for the given grid element basedat least in part on the difference between (i) the assignedcharacteristic grid-element depth value for the given grid element and(ii) an offset depth value corresponding to a depth between the 3D depthsensor and a front plane of the shipping container. Thus, if the depthsensor registers an absolute value of, e.g., 7 meters as a depth valuefor a given point or grid element and it is pre-provisioned or runtimedetermined that the depth sensor is 4 meters from the front plane of theopen end of the shipping container, the computing system 600 mayconsider the unloaded-container-portion depth value for that gridelement to be 3 meters. And certainly numerous other examples could belisted.

In some cases, the depth dimension of the shipping container that isused to derive a loaded-container-portion depth value from anunloaded-container-portion depth value for a given grid element is agrid-element-specific depth dimension that is based on a correspondinggrid element in a reference empty-container point cloud. As describedabove, the back wall could be flat or curved, as depicted in FIGS. 2Aand 2B, and the grid-element-specific depth dimension for a given gridelement could accordingly reflect this. A reference point cloud could begathered using an empty shipping container of the same type, and thatreference point cloud could be stored in data storage and recalled,perhaps on a grid-element-by-grid-element basis to perform theherein-described calculations.

At step 708, the computing system 600 calculates aloaded-container-portion volume of the shipping container by aggregatingthe respective loaded-container-portion grid-element volumes that werecalculated at step 706, giving a result that corresponds to what volume(in, e.g., cubic meters) of the shipping container has been loaded. Itis noted that loaded in this context essentially means no longeravailable for loading. Thus, empty space that is now inaccessible due topackages being stacked in the way would be counted as loaded right alongwith space in the shipping container that is actually occupied by agiven package.

At step 710, the computing system 600 calculates an estimated fullnessof the shipping container based on (i) the loaded-container-portionvolume that was calculated at step 708 and (ii) a capacity of theshipping container. In particular, the estimated fullness of theshipping container may be calculated as the loaded-portion volume of theshipping container divided by the capacity of the shipping container.The capacity of the shipping container could be determined in multipledifferent ways, some of which are described below in connection withFIG. 8.

In one embodiment, the computing system 600 determines the capacity ofthe shipping container based at least in part on the received 3D pointcloud. Thus, the 3D point cloud may be indicative of the dimensions ofthe shipping container such that the capacity of the shipping containercan be determined. In another embodiment, the computing system receivesan optical image of the shipping container, and determines the capacityof the shipping container based at least in part on the received opticalimage. This could include determining actual dimensions of the shippingcontainer from the optical image, and could instead or in additioninclude extracting an identifier of the shipping container from theoptical image, perhaps using optical character recognition (OCR), andthen querying a local or remote database using that identifier in orderto retrieve dimension and/or capacity data pertaining to the particularshipping container.

It is noted that, in some embodiments, the system may determine that theentire interior of the shipping container is not visible to the depthsensor, perhaps due to the relative location and arrangement of thedepth sensor and the shipping container. In such instances, the systemmay define a volume of interest (VOI) as being the part of the interiorof the container that is visible to the depth sensor. The system may insome such instances calculate the estimated fullness of the container tobe loaded portion of the VOI divided by the capacity (i.e., totalvolume) of the VOL In other embodiments, the system may simply assumethat any internal portion of the shipping container that cannot be seenwith the depth camera is loaded, and in such cases may still calculatethe estimated fullness as the loaded portion of the entire shippingcontainer divided by the total capacity of the entire shippingcontainer. And certainly other example implementations could be listedhere as well.

At step 712, the computing system 600 outputs the calculated estimatedfullness of the shipping container, perhaps to a display, perhaps to adata storage, perhaps using wireless and/or wired communication totransmit the calculated estimated fullness of the shipping container toone or more other devices or systems, and/or perhaps to one or moreother destinations.

FIG. 8 depicts a shipping container having an optically readableidentifier in accordance with some embodiments. In particular, FIG. 8depicts a container 802, an indicia 804 (e.g., bar code or alphanumericidentifier), and an optical reader 806. There are several differenttypes of optical readers 806 that may be used, such as a barcodescanner, a camera, and/or the like. In one embodiment, the opticalreader 806 acquires an alphanumeric identifier of the container datausing OCR. The computing system may then use that acquired alphanumericidentifier of the container to query a database for dimension datapertaining to the shipping container. And certainly other exampleimplementations could be listed here as well.

In the foregoing specification, specific embodiments have beendescribed. However, one of ordinary skill in the art appreciates thatvarious modifications and changes can be made without departing from thescope of the invention as set forth in the claims below. Accordingly,the specification and figures are to be regarded in an illustrativerather than a restrictive sense, and all such modifications are intendedto be included within the scope of present teachings.

The benefits, advantages, solutions to problems, and any element(s) thatmay cause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeatures or elements of any or all the claims. The invention is definedsolely by the appended claims including any amendments made during thependency of this application and all equivalents of those claims asissued.

Moreover in this document, relational terms such as first and second,top and bottom, and the like may be used solely to distinguish oneentity or action from another entity or action without necessarilyrequiring or implying any actual such relationship or order between suchentities or actions. The terms “comprises,” “comprising,” “has”,“having,” “includes”, “including,” “contains”, “containing” or any othervariation thereof, are intended to cover a non-exclusive inclusion, suchthat a process, method, article, or apparatus that comprises, has,includes, contains a list of elements does not include only thoseelements but may include other elements not expressly listed or inherentto such process, method, article, or apparatus. An element proceeded by“comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . .a” does not, without more constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises, has, includes, contains the element. The terms“a” and “an” are defined as one or more unless explicitly statedotherwise herein. The terms “substantially”, “essentially”,“approximately”, “about” or any other version thereof, are defined asbeing close to as understood by one of ordinary skill in the art, and inone non-limiting embodiment the term is defined to be within 10%, inanother embodiment within 5%, in another embodiment within 1% and inanother embodiment within 0.5%. The term “coupled” as used herein isdefined as connected, although not necessarily directly and notnecessarily mechanically. A device or structure that is “configured” ina certain way is configured in at least that way, but may also beconfigured in ways that are not listed.

It will be appreciated that some embodiments may be comprised of one ormore generic or specialized processors (or “processing devices”) such asmicroprocessors, digital signal processors, customized processors andfield programmable gate arrays (FPGAs) and unique stored programinstructions (including both software and firmware) that control the oneor more processors to implement, in conjunction with certainnon-processor circuits, some, most, or all of the functions of themethod and/or apparatus described herein. Alternatively, some or allfunctions could be implemented by a state machine that has no storedprogram instructions, or in one or more application specific integratedcircuits (ASICs), in which each function or some combinations of certainof the functions are implemented as custom logic. Of course, acombination of the two approaches could be used.

Moreover, an embodiment can be implemented as a computer-readablestorage medium having computer readable code stored thereon forprogramming a computer (e.g., comprising a processor) to perform amethod as described and claimed herein. Examples of suchcomputer-readable storage mediums include, but are not limited to, ahard disk, a CD-ROM, an optical storage device, a magnetic storagedevice, a ROM (Read Only Memory), a PROM (Programmable Read OnlyMemory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM(Electrically Erasable Programmable Read Only Memory) and a Flashmemory. Further, it is expected that one of ordinary skill,notwithstanding possibly significant effort and many design choicesmotivated by, for example, available time, current technology, andeconomic considerations, when guided by the concepts and principlesdisclosed herein will be readily capable of generating such softwareinstructions and programs and ICs with minimal experimentation.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

We claim:
 1. A method comprising: receiving a three-dimensional (3D)point cloud from a depth sensor that is oriented towards an open end ofa shipping container, the point cloud comprising a plurality of pointsthat each have a respective depth value; segmenting the received 3Dpoint cloud among a plurality of grid elements; calculating a respectiveloaded-container-portion grid-element volume for each grid element basedon at least a respective grid-element area and a respectiveloaded-container-portion grid-element depth value for each respectivegrid element that is determined based at least in part on the differencebetween (i) a depth dimension of the shipping container and (ii) anunloaded-container-portion depth value for the corresponding gridelement; calculating a loaded-container-portion volume of the shippingcontainer by aggregating the calculated respectiveloaded-container-portion grid-element volumes; calculating an estimatedfullness of the shipping container based on the loaded-container-portionvolume and a capacity of the shipping container; and outputting thecalculated estimated fullness of the shipping container.
 2. The methodof claim 1, wherein the plurality of grid elements collectively forms atwo-dimensional (2D) grid image that corresponds to a plane that isparallel to the open end of the shipping container, each grid elementhaving a respective grid-element area.
 3. The method of claim 1, whereindetermining the unloaded-container-portion depth value for a given gridelement comprises: assigning a grid-element depth value to the givengrid element based on the depth values of the points in the point cloudthat correspond to the given grid element; and determining theunloaded-container-portion depth value for the given grid element basedat least in part on the difference between (i) the assigned grid-elementdepth value for the given grid element and (ii) an offset depth valuecorresponding to a depth between the depth sensor and a front plane ofthe shipping container.
 4. The method of claim 3, wherein assigning thegrid-element depth value for the given grid element based on the depthvalues of the points in the point cloud that correspond to the givengrid element comprises assigning as the grid-element depth value for thegiven grid element a minimum value from among the depth values of thepoints in the point cloud that correspond to the given grid element. 5.The method of claim 3, wherein assigning the grid-element depth valuefor the given grid element based on the depth values of the points inthe point cloud that correspond to the given grid element comprisesassigning as the grid-element depth value for the given grid element anaverage value of the depth values of the points in the point cloud thatcorrespond to the given grid element.
 6. The method of claim 1, whereinthe depth dimension of the shipping container is a grid-element-specificdepth dimension that is based on a corresponding grid element in areference empty-container point cloud.
 7. The method of claim 6, whereinthe reference empty-container point cloud reflects a back wall of theshipping container being a flat surface.
 8. The method of claim 6,wherein the reference empty-container point cloud reflects a back wallof the shipping container being a curved surface.
 9. The method of claim2, further comprising cleaning up the 2D grid image prior to determininga respective loaded-container-portion grid-element depth value for eachgrid element.
 10. The method of claim 1, wherein the depth sensor has anoptical axis and an image plane, the method further comprising: prior tosegmenting the received point cloud among the plurality of gridelements, rotating the received 3D point cloud to align (i) the opticalaxis with a ground level and (ii) the image plane with an end plane ofthe shipping container.
 11. The method of claim 10, wherein the rotatingthe point cloud is based on an offline calibration process using theground level and the end plane as reference.
 12. The method of claim 1,further comprising determining the capacity of the shipping containerbased at least in part on the received 3D point cloud.
 13. The method ofclaim 1, further comprising: receiving an optical image of the shippingcontainer; and determining the capacity of the shipping container basedat least in part on the received optical image.
 14. The method of claim13, wherein determining the capacity of the shipping container based atleast in part on the received optical image comprises: determining atleast one physical dimension of the shipping container from the receivedoptical image; and determining the capacity of the shipping containerbased on the at least one determined physical dimension.
 15. The methodof claim 13, wherein determining the capacity of the shipping containerbased at least in part on the received optical image comprises: usingoptical character recognition (OCR) on the at least one received opticalimage to ascertain at least one identifier of the shipping container;and using the at least one ascertained identifier of the shippingcontainer to determine the capacity of the shipping container.
 16. Themethod of claim 1, wherein each grid element has sides substantiallyequal to 5 millimeters (mm) in length.
 17. The method of claim 1,wherein each grid element is substantially square in shape, and whereina grid-element side length is an adjustable parameter.
 18. A systemcomprising: a depth sensor that is oriented towards an open end of ashipping container; a communication interface; a processor; and datastorage containing instructions executable by the processor for causingthe system to carry out a set of functions, the set of functionsincluding: receiving a three-dimensional (3D) point cloud from the depthsensor, the point cloud comprising a plurality of points that each havea respective depth value; segmenting the received 3D point cloud among aplurality of grid elements; calculating a respectiveloaded-container-portion grid-element volume for each grid element basedon at least a respective grid-element area and a respectiveloaded-container-portion grid-element depth value for each respectivegrid element that is determined based at least in part on the differencebetween (i) a depth dimension of the shipping container and (ii) anunloaded-container-portion depth value for the corresponding gridelement; calculating a loaded-container-portion volume of the shippingcontainer by aggregating the calculated respectiveloaded-container-portion grid-element volumes; calculating an estimatedfullness of the shipping container based on the loaded-container-portionvolume and a capacity of the shipping container; and outputting thecalculated estimated fullness of the shipping container.
 19. The systemof claim 18, wherein: the plurality of grid elements collectively formsa two-dimensional (2D) grid image that corresponds to a plane that isparallel to the open end of the shipping container, each grid elementhaving a respective grid-element area.
 20. A method comprising:receiving a three-dimensional (3D) point cloud from a depth sensor thatis oriented towards an open end of a shipping container, the point cloudcomprising a plurality of points that each have a respective depthvalue, wherein the depth sensor has an optical axis and an image plane;rotating the received 3D point cloud to align (i) the optical axis witha ground level and (ii) the image plane with an end plane of theshipping container; segmenting the rotated 3D point cloud among aplurality of grid elements that collectively form a two-dimensional (2D)grid image that corresponds to a plane that is parallel to the open endof the shipping container, each grid element having a respectivegrid-element area; determining a respective loaded-container-portiongrid-element depth value for each grid element at least in part bycomparing (i) the respective portion of the projected point cloud thatis overlaid by the respective grid-element area of the respective gridelement with (ii) a respective corresponding portion of a referenceempty-container point cloud; calculating a respectiveloaded-container-portion grid-element volume for each grid element basedon at least the respective grid-element area and the respectiveloaded-container-portion grid-element depth value for each respectivegrid element; calculating a loaded-container-portion volume of theshipping container by aggregating the calculated respectiveloaded-container-portion grid-element volumes; determining a capacity ofthe shipping container based at least in part on the received depth datacalculating an estimated fullness of the shipping container based on theloaded-container-portion volume and the determined capacity of theshipping container; and outputting the calculated estimated fullness ofthe shipping container.